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1      Introduction 

The  recovery  of  torpedoes  on  the  various  NUWES  test  ranges  is  an  im- 
portant problem  to  the  U.S.  Navy.  The  recovery  operation  is  carried  on 
undersea  with  the  aid  of  digging  equipment  and  underwater  video  cam- 
era equipped  with  artificial  lights.  The  digging  operation  is  controlled  and 
monitored  on  video  monitors  at  the  surface  on  board  the  recovery  vessel. 

The  recovery  of  torpedoes  is  not  a  simple  problem.  Torpedoes  are  some- 
times buried  in  the  sediment  and  are  difficult  to  locate.  In  addition,  the 
recovery  equipment  in  its  attempt  to  dig  out  the  torpedo  stirs  up  sediment 
which  visually  obscures  the  object  of  interest  and  impedes  the  recovery 
operation.  Since  the  human  operator  carrying  on  the  recovery  operation 
relies  heavily  on  the  video  image  received  from  the  underwater  camera  and 
displayed  on  the  monitor,  improving  the  quality  and  intelligibility  of  the 
video  image  in  this  turbid  water  viewing  environment  has  the  potential  to 
significantly  increase  the  efficiency  of  the  difficult  recovery  operation. 

An  effort  to  process  images  in  turbid  water  viewing  environment  for 
possible  improvement  in  image  quality  and  intelligibility  is  in  progress  at 
the  Naval  Postgraduate  School  under  the  direction  of  Professor  Charles 
Therrien.  As  a  part  of  this  effort,  over  the  past  six  weeks  we  investigated 
the  performance  and  computational  requirements  of  an  existing  algorithm. 
We  also  developed  new  methods,  investigated  their  computational  require- 
ments, and  studied  their  expected  performance.  Finally,  we  performed 
some  very  preliminary  study  on  issues  related  to  the  possible  real  time  im- 
plementation of  these  algorithms.  In  this  report,  we  describe  the  results  of 
these  efforts. 


2      Adaptive  Filtering 
2.1      Basic  Algorithm 

Observation  of  video  images  in  a  turbid  water  environment  shows  that  the 
contrast  of  images  is  often  reduced  significantly  for  a  variety  of  reasons 
including  the  presence  of  sediment  that  is  stirred  in  a  digging  operation. 
One  approach  that  was  previously  considered  at  the  Naval  Postgraduate 
School  and  which  we  explored  further  as  a  part  of  this  effort  is  to  apply  a 
contrast  enhancement  algorithm.  The  specific  algorithm  used  is  shown  in 
Figure  1. 

In  the  figure,  /(nx,n2)  is  a  frame  (N  x  N  pixels)  of  the  video  data, 
/l(»*i,  ri2)  is  the  local  luminance  mean  which  is  obtained  by  low-pass  filter- 
ing /(ni,n2),  and  ///(ni,n2)  is  the  local  contrast  obtained  by  subtracting 
/£,(n1,n2)  from  /(n1?  ra2).  The  low-pass  filtering  operation  is  performed 
by  local  averaging  over  an  M  x  M  pixel  region.  The  processed  local  con- 
trast ///(ni,n2)  is  obtained  by  multiplying  /#(ni,n2)  by  the  contrast  en- 
hancement factor  k{fi).  The  processed  local  luminance  mean  /l(^i,^2)  is 
obtained  by  applying  a  point  non-linearity  function  to  /l(^i,  ^2)-  The  pro- 
cessed image  p(ni,ri2)  is  obtained  by  combining  /tf(n!,n2)  and  /£,(?*!,  n2). 

The  algorithm  in  Figure  1  is  capable  of  modifying  the  local  contrast 
as  a  function  of  the  local  luminance  mean  and  modifying  the  local  lumi- 
nance mean.  The  algorithm  has  been  [1]  successfully  applied  previously  to 
improving  the  contrast  of  optical  images  taken  from  an  airplane  through 
varying  amounts  of  cloud  cover.  Since  degradations  such  as  those  due  to 
the  sediment  stirred  up  during  the  recovery  operation  appear  to  reduce  the 
image  contrast  in  a  manner  similar  to  the  contrast  reduction  due  to  cloud 
cover,  this  particular  algorithm  was  chosen  originally  for  a  more  detailed 
study. 
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2.2  Parameter  Choice  and  Expected  Performance 

The  algorithm  in  Figure  1  has  been  applied  to  a  limited  set  of  data  from 
NUWES.  From  this  study,  we  have  observed  that  the  data  generally  have  a 
certain  level  of  background  noise  and  this  limits  the  utility  of  the  adaptive 
filtering  algorithm.  Any  contrast  enhancement  method  including  the  adap- 
tive filtering  method  we  have  studied  tends  to  emphasize  high  frequency 
components  and  as  a  result  tends  to  emphasize  the  background  noise.  For 
the  video  data  from  NUWES,  we  have  observed  that  the  background  noise 
becomes  very  visible  when  the  contrast  enhancement  factor  &(/l)  is  chosen 
above  3.  This  is  in  sharp  contrast  with  the  cloud  cover  data  to  which  this 
algorithm  was  originally  applied.  The  noise  level  in  the  cloud  cover  data 
was  sufficiently  low  so  that  the  contrast  enhancement  factor  in  the  range 
of  6  ~  8  could  be  used  without  any  visible  noise  in  the  processed  image. 

Based  on  the  above  considerations,  the  choice  of  &(/£,)  and  the  nonlin- 
earity  function  we  recommend  is  shown  in  Figures  2  and  3.  The  function 
&(/l)  hes  between  1.0  and  3.0,  and  increases  as  fi  increases.  This  feature 
exploits  the  notion  that  the  same  level  of  noise  is  more  visible  in  the  dark 
regions  relative  to  the  bright  regions.  We  also  recommend  a  mask  size  of 
5x5  pixels  for  the  low-pass  filtering  operation. 

Since  the  contrast  enhancement  factgor  A:(/^)  recommended  is  less  than 
3,  a  very  large  amount  of  contrast  enhancement  is  not  possible.  However, 
the  contrast  enhancement  by  a  factor  of  2  ~  3  is  still  significant  and  we 
expect  that  noticeable  contrast  enhancement  is  still  possible  with  the  rec- 
ommended choice  of  parameters. 

2.3  Computational  Requirements 

All  our  discussions  on  computational  requirements  in  this  and  future  sec- 
tions are  based  on  the  assumption  that  /(r»i,  712)  is  an  image  frame  of  N  x  N 
pixels,  the  low-pass  filtering  operation  is  a  simple  average  over  a  region  of 
M  x  M  pixels,  and  R  frames  are  processed  per  second. 

The  major  computations  involved  in  processing  one  image  frame  using 
the  adaptive  filtering  algorithm  are  as  follows: 
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Additions 

Multiplications 

Table  Look-ups 

Low-pass  filtering 

M2N2 

N2 

Subtraction  of  /i(ni ,  n?) 

N2 

Hh) 

N2 

Multiplication  with  fc(/t) 

N2 

Nonlinearity 

N2 

Addition  of  fn («i,  "2) 

and  /L(ni,n2) 

N2 

Total 

(M2  +  2)N2 

IN2 

2N2 

From  the  above  result, 

Computations/ sec:       (M2  +  2)N2R  additions,  2N2R  multiplications, 
2N2R  table  look-up  operations 

As  an  example,  when  N  =  512,  M  =  5,  and  R  =  30  (full  video  rate),  we 
require  approximately  200M  additions,  15M  multiplications,  and  15M  table 
look-up  operations  per  second.  As  another  example,  when  N  =  512,  M  =  5, 
and  R  =  1  (one  frame/sec),  we  require  approximately  7M  additions,  |M 
multiplications  and  |M  table  look-up  operations  per  second. 

It  may  be  possible  to  reduce  computations  that  arise  from  the  low-pass 
filtering  operation,  which  is  the  major  computational  requirement  in  the 
algorithm.  The  low-pass  filtered  version  /l(^i,^2)  Qas  a  low-pass  charac- 
teristic and  we  may  be  able  to  undersample  it.  If  we  undersample  /l(^i,  n2) 
by  a  factor  of  S  x  5,  we  can  reduce  the  computations  required  in  the  low- 
pass  filtering  operation  by  a  factor  of  S2  at  the  expense  of  an  increase  of 
approximately  iV2  additions  and  N2  multiplications  per  frame  required  for 
interpolation  of  /iX^i,^)-  There  will  be  some  decrease  in  performance, 
but  the  level  of  performance  decrease  is  not  expected  to  be  too  serious. 
We  recommend  that  future  study  include  a  study  on  this  trade-off  between 
performance  decrease  and  reduction  in  computations. 

To  determine  the  storage  requirements,  we  assume  that  the  video  data 
are  raster- scanned  as  in  the  NTSC  signal.  We  also  assume  that  one  memory 
unit  contains  one  pixel  intensity,  which  is  typically  represented  with  8  bits 
(1  byte).  The  storage  requirements  are  as  follows: 


/(ni,n2) 

/L(ni,n2) 

Kft) 

Nonlinearity 


MiV  memory  units 
MN  memory  units 
256  memory  units 
256  memory  units 


Total 


2MN  +  512  memory  units 


From  the  above  result, 

Storage  Requirements 


2MN  +  512  memory  units 


For  N  =  512  and  M  =  5,  the  algorithm  requires  about  6K  memory 
units.  This  memory  requirement  is  not  an  issue  in  the  real  time  implemen- 
tation of  the  adaptive  filtering  algorithm. 


3     Modified  Adaptive  Filtering 
3.1      Basic  Algorithm 

To  account  for  the  noise  visibility  problem  associated  with  the  adaptive 
filtering  method,  one  approach  proposed  and  studied  by  Franco  [2]  is  to 
enhance  the  contrast  using  the  adaptive  filtering  algorithm  and  then  to 
apply  a  noise  reduction  system  to  reduce  the  background  noise  that  has 
been  accentuated.  Even  though  this  approach  is  interesting,  it  has  several 
problems.  In  a  typical  noise  reduction  system,  the  local  luminance  mean 
/£,(ni,n2)  is  often  computed  and  cascading  the  adaptive  filtering  method 
with  a  noise  reduction  system  requires  computation  of  /l(«i,^2)  twice.  In 
addition,  a  noise  reduction  system  often  requires  an  estimate  of  the  noise 
variance.  The  noise  variance  depends  on  the  contrast  enhancement  factor 
k{f£)  used  in  the  adaptive  filtering  algorithm,  and  this  complicates  the 
design  of  the  noise  reduction  system. 

A  simple  method  which  in  a  sense  integrates  contrast  enhancement  with 
noise  reduction  has  been  developed.  We'll  refer  to  this  method  as  modified 
adaptive  filtering  algorithm.  The  algorithm  is  shown  in  Figure  4.  The 
main  difference  between  this  algorithm  and  the  adaptive  filtering  method 
discussed  in  Section  2  is  that  the  contrast  enhancement  factor  k  is  now  a 
function  of  the  local  luminance  mean  /&  and  the  local  variance  a2.  The 
local  variance  can  be  computed  approximately  from  /#(ni,n2)  by 

cr2f(n1,n2)  =  —       £  £       f2H{kuk2) 

*1  — 11 —  «2  =T12 — 

where  L  is  assumed  to  be  an  odd  integer  and  the  region  over  which  a2  is 
computed  has  size  of  L  x  L  pixels.  The  normalization  factor  L2  can  be 
accounted  for  in  determining  k(fL,a2)  and  therefore  can  be  ignored. 

When  a2  is  small,  the  area  is  likely  to  correspond  to  a  region  with  little 
signal  component  (uniform  background  region)  and  contrast  enhancement 
in  the  region  is  likely  to  boost  only  the  background  noise.  When  a2  is  very 
large,  the  signal  component  may  have  large  energy  and  a  very  large  contrast 
enhancement  may  not  be  necessary.  When  a2  is  in  the  mid-range,  a  weak 
signal  component  is  likely  to  be  present  and  a  large  contrast  enhancement 

10 


■Pcnvn») 

v 

,-f>'A) 

A. 

p(V 

1 

/ 

9 

» 

"V 

'¥ 

Loo-pfc$s 

Varr^yvOL 

ftCf.;^) 

>  '^to»W 

/ 

^       / 

d 

SU^ni) 

NJoAliVvJeAh^ 

A 

/ 

/ 

fiW 

Ti'qur^    f.      MoJ;ff-ei     ficUpt*Vt.  ftibsrtAjl     AJWthm, 


11 


is  likely  to  be  beneficial.  Even  though  the  noise  in  this  region  will  be 
accentuated,  the  region  involved  is  only  a  fraction  of  the  image  frame  and 
the  presence  of  the  accentuated  signal  component  can  mask  the  noise  to 
some  extent.  The  dependence  of  k(fL,cr2)  on  a2  allows  us  to  modify  the 
contrast  as  a  function  of  a2. 

3.2  Parameter  Choice  and  Expected  Performance 

The  algorithm  in  Figure  4  is  currently  being  implemented  by  Lieutenant 
Roberto  Ventura.  At  the  time  of  this  report,  we  do  not  have  results  of 
applying  this  method  to  the  data  from  NUWES.  It  is  expected,  however, 
to  solve  the  noise  visibility  problem  associated  with  the  adaptive  filtering 
method  to  some  extent. 

Choosing  the  parameters  of  the  algorithm  requires  careful  evaluation 
of  the  processed  images.  As  an  initial  starting  point,  we  recommend  the 
parameter  choice  shown  in  Figures  5  and  6.  The  choice  of  a  separable 
function  &(/l,  cr))  =  fci(/L)^2(c/)  is  due  to  the  desire  to  decouple  the  effects 
of  fi  and  a2  on  k(fL,cr2)  and  reduction  in  the  required  storage.  If  storage 
is  not  a  problem,  k(fi,<72)  can  be  precomputed  and  stored.  The  choice  of 
&i(/l)  is  based  on  the  same  considerations  as  in  Section  2.2.  The  choice 
of  k2(cr2f)  is  chosen  based  on  several  considerations.  When  a2  is  small, 
there  is  likely  to  be  little  signal  component  and  large  contrast  enhancement 
will  only  result  in  noise  enhancement.  When  o2,  is  very  large,  the  signal 
component  is  already  very  strong  and  there  is  not  much  need  for  a  large 
contrast  increase. 

3.3  Computational  Requirements 

The  modified  adaptive  filtering  method  in  Figure  4  is  more  expensive  com- 
putationally than  the  adaptive  filtering  method  in  Figure  1.  The  additional 
computations  per  second  are 

Computation  of  Variance:     L2N2R  multiply/adds. 

Computation  of  k(fL,<jj):     N2R  multiplications,  N2R  table  lookups. 

Combining  this  result  with  the  results  in  Section  2.3, 

Computations/sec:  (M2  +  2)N2R  additions,  3  N2R  multiplications, 
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L2N2R  multiply/adds,  3N2R  table  look-up  operations 

If  we  assume  that  L  =  M  and  one  multiply/add  operation,  one  add 
operation,  and  one  multiply  operation  take  the  same  computation  time,  the 
computational  requirements  of  the  modified  adaptive  filtering  algorithm  is 
approximately  twice  that  of  the  adaptive  filtering  algorithm. 

The  storage  requirements  are  essentially  the  same  as  that  of  the  adaptive 
method.  The  small  increase  is  due  to  the  storage  of  &2 (<?/)•  The  storage 
requirements  are 

Storage  Requirements:  2MN  +  768  memory  units 
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4     Temporal  Filtering 

4.1      Basic  Algorithm 

Careful  observation  of  the  video  data  from  NUWES  shows  that  some  form 
of  frame  averaging  may  result  in  image  enhancement.  Specifically,  in  a  typ- 
ical torpedo  recovery  operation,  the  underwater  camera  is  often  stationary 
or  moves  slowly.  As  a  result,  objects  of  interest  such  as  a  torpedo  displayed 
on  the  monitor  do  not  appear  to  change  much  from  one  frame  to  the  next 
frame.  However,  the  elements  such  as  water  bubbles  and  stirred-  up  sedi- 
ment that  degrade  the  visibility  of  objects  of  interest  appear  to  change  quite 
rapidly.  Some  form  of  frame  averaging,  therefore,  has  potential  to  add  the 
signal  component  constructively  while  adding  signals  from  the  degrading 
sources  destructively.  This  can  result  in  improvement  of  the  signal  to  noise 
ratio  (SNR). 

Frame  averaging  can  involve  storage  of  the  frames  involved,  which  can 
increase  the  storage  requirements  drastically.  One  method  that  performs 
frame  averaging  without  requiring  storage  of  more  than  one  frame  at  a  time 
is  first  order  recursive  temporal  filtering.  Consider  one  particular  pixel.  Let 
f(n)  denote  the  intensity  of  the  image  at  the  nth  frame  at  that  particular 
pixel.  Note  that  the  variable  "n"  is  a  time  variable,  not  a  spatial  variable. 
The  processed  image  p(n)  in  the  first  order  recursive  temporal  filtering  is 
given  by 

p(n)  =  (1  -  a)  •  p(n  -  1)  +  a  •  /(n), 

where  "a"  is  a  constant  between  0  and  1.  As  /(n)  is  received  and  once 
p(n)  is  computed,  p(n  —  1)  is  no  longer  needed  and  can  be  replaced  with 
p(n).  At  any  given  time,  therefore,  we  need  to  store  only  one  frame  of  data. 
The  system  function  H(z)  and  unit  sample  response  of  the  system  h(n)  are 
given  by 

H(z)  = ,    a     -     - 

and 

h(n)  =  <z(l  —  a)nu(n). 

As  "a"  approaches  0,  many  frames  are  averaged.  When  a  =  1,  no  temporal 
filtering  is  performed. 
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Many  variations  of  the  above  recursive  filtering  method  are  possible. 
For  example,  by  initializing  p(n)  once  in  a  while,  choosing  "a"  close  to 
zero,  and  normalizing  the  result,  a  result  essentially  the  same  as  simple 
FIR  filtering  can  be  obtained.  We  also  note  that  the  contrast  enhancement 
methods  discussed  in  Sections  2  and  3  can  also  be  applied  to  the  temporally 
filtered  images,  if  desired. 

4.2  Parameter  Choice  and  Expected  Performance 

A  sequence  of  image  frames  that  have  been  digitized  are  not  available  and 
we  were  not  able  to  process  the  data  using  the  temporal  filtering  algorithm 
discussed  above.  To  the  extent  that  our  assumption  that  objects  of  inter- 
est do  not  change  rapidly  between  consecutive  frames  while  the  degrading 
sources  do  is  valid,  the  algorithm  is  expected  to  perform  very  well. 

To  see  the  expected  performance  improvement  in  an  ideal  environment, 
suppose  /(n)  can  be  expressed  as 

/(n)  =  s(n)  +  w(n) 

where  s(n)  is  the  signal  which  is  constant  independent  of  n  and  w(n)  is 
zero- mean  white  noise  with  variance  of  a2w.  The  processed  image  p(n)  can 
be  expressed  as 

p(n)  =  s(n)  +  wp(n) 

where  wp(n)  is  zero-mean  white  (in  the  spatial  domain)  noise  with  vari- 
ance of  1  ,°_,2  •  <7^.  When  a  =  0.2,  the  noise  variance  reduction  is  by 
approximately  a  factor  of  10,  corresponding  to  lOdB  SNR  improvement. 

In  practice,  of  course,  the  assumptions  made  in  the  above  analysis  will 
not  be  valid.  The  signal,  for  example,  will  change  as  a  function  of  time 
and  this  will  cause  signal  blurring.  There  is  potential  for  significant  image 
enhancement,  however,  and  we  recommend  that  future  studies  include  the 
application  of  temporal  filtering  to  the  data  from  NUWES.  Our  initial 
recommendation  for  the  choice  of  "a"  is  0.2. 

4.3  Computational  Requirements. 

The  computations  involved  are  N2  additions  and  2N2  multiplications  per 
frame.  Therefore, 
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Computations/ sec:  N2R  additions,  2N2R  multiplications 

When  N  =  512  and  R  =  30,  the  computational  requirements  are  l\M 
additions  and  15M  multiplications  per  second. 

The  storage  requirement  is  the  storage  of  one  image  frame  and  therefore, 

Storage  requirement:  N2  memory  units 
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5     Pseudo  Color  Representation 

It  is  well  known  that  the  human  visual  system  is  quite  sensitive  to  color. 
The  number  of  distinguishable  intensities,  for  example,  is  much  less  than 
the  number  of  distinguishable  colors  and  intensities.  As  a  result,  when  a 
black  and  white  image  is  displayed  using  color,  the  result  may  not  be  a 
natural  looking  image  but  the  contrast  of  the  image  may  be  considerably 
improved.  In  addition,  color  images  are  generally  much  more  pleasant  to 
look  at  than  black  and  white  images. 

Mapping  a  black  and  white  image  /(n!,n2)  to  a  color  image  involves 
generation  of  the  red  component  //*(ni,  7^2),  the  green  component  /g("i,  "2) 
and  the  blue  component  /s(ni,n2).  Designing  the  specific  transformation 
that  maps  f(rii,n2)  to  /u(ni,fl2)>  /b(ni>n2)  and  fB{n\in2)  is  limited  only 
by  one's  artistic  imagination,  and  involves  a  fair  amount  of  trial  and  error. 
One  recommendation  we  have  for  the  design  of  the  transformation  table 
is  that  similar  colors  be  used  for  similar  pixel  intensities.  Otherwise,  the 
resulting  color  image  will  appear  noisy.  Another  recommendation  is  that 
low  intensities  be  mapped  to  blue  and  high  intensities  be  mapped  to  red. 
The  intensities  in  between  can  perhaps  be  mapped  following  the  rainbow 
colors.  It  is  well  known  that  people  perceive  blue  as  "cold"  or  "dark"  and 
perceive  red  as  "hot"  or  "bright". 

Once  the  transformation  table  is  designed,  processing  involves  only  N2R 
table  look-up  operations  per  second  and  storage  requirement  for  the  table 
is  minimal.  Allowing  table  look-up  operations  prior  to  image  display  is  a 
common  feature  in  commercially  available  real  time  video  data  digitization 
and  display  systems.  Even  though  the  level  of  performance  improvement 
is  not  clear,  it  is  a  worthwhile  approach  to  be  explored  in  future  study. 
Pseudo- color  operation,  of  course  can  be  performed  in  addition  to  other 
methods  discussed  in  previous  sections. 
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6     Hardware  Implementation 

To  apply  the  algorithms  discussed  in  previous  sections  to  enhance-  the  ef- 
ficiency of  the  torpedo  recovery  operation,  their  real  time  implementation 
is  essential.  In  this  section  we  discuss  some  very  preliminary  studies  we 
performed  to  determine  the  feasibility  of  implementing  the  algorithms  in 
real  time  at  a  reasonable  cost. 

One  cost-effective  approach  to  real  time  implementation  is  to  interface 
a  signal  processing  (SP)  chip  to  a  personal  computer  (PC)  which  has  been 
interfaced  to  a  unit  such  as  the  PC  Vision  board  that  is  capable  of  digitizing 
and  displaying  the  video  data  in  real  time,  as  shown  in  Figure  7.  Using  a  PC 
as  an  overall  control  unit  and  interfacing  an  image  digitization  and  display 
unit  such  as  the  PC  Vision  board  is  a  very  cost-effective  way  to  acquire 
and  display  video  data.  The  algorithms  we  wish  to  implement  in  real  time 
require  a  very  large  number  of  arithmetic  operations  and  the  SP  chip  is 
necessary  to  perform  the  arithmetic  operations  very  fast.  The  adaptive 
filtering  method  discussed  in  Section  2  has  already  been  implemented  [3] 
using  the  approach  in  Figure  7.  With  a  NEC  J/PD77230  chip  and  PC-9800, 
processing  one  image  frame  using  the  adaptive  filtering  method  discussed 
in  Section  2  with  N=256  and  M=17  requires  approximately  3  seconds. 

In  studying  the  feasibility  of  real  time  implementation,  a  number  of  is- 
sues have  to  be  considered.  In  this  very  preliminary  study,  however,  we 
have  made  a  variety  of  simplifying  assumptions.  Specifically,  we  assume 
that  storage  requirement  is  not  an  issue  in  real  time  implementation.  Since 
the  storage  requirement  for  data  is  in  the  order  of  iV2  memory  units  or 
less,  this  assumption  is  reasonable.  We  have  also  assumed  that  the  compu- 
tational time  required  for  each  of  an  add,  a  multiply,  and  a  multiply/add 
operation  is  the  same.  As  floating  point  arithmetic  operations  are  becoming 
more  common,  this  assumption  also  appears  reasonable.  We'll  refer  to  an 
addition,  a  multiplication,  or  a  multiply/add  operation  as  one  arithmetic 
operation.  We  denote  the  computation  time  for  one  arithmetic  operation 
by  c  sec.  For  currently  available  SP  chips  such  as  Texas  Instrument's  most 
recent  TMS320  family  chip,  the  computation  time  "c"  is  in  the  order  of  100 
nanosec  (10~9  sec).  We  also  assume  that  the  computation  time  required 
for  all  arithmetic  operations  is  a  fraction  (p%)  of  the  total  amount  of  time 
required  for  a  given  algorithm.  The  fraction  "p"  %  depends  on  a  number 
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of  factors  such  as  data  transfer  time,  algorithm  complexity,  the  specific  SP 
chip,  PC,  and  interface  used.  We'll  consider  two  different  reasonable  values 
of  p  in  our  analysis. 

Under  the  assumptions  and  notations  discussed  above,  we  have  com- 
puted the  amount  of  time  required  for  processing  1  second  of  video  data 
for  various  choices  of  the  parameters.  The  results  are  shown  in  Table  1.  If 
the  required  processing  time  is  less  than  1  second,  it  implies  that  real  time 
implementation  is  possible.  For  the  specific  choices  of  the  parameters  used 
in  the  table,  none  of  the  cases  can  be  implemented  in  real  time.  However, 
some  of  the  cases  such  as  temporal  filtering  are  not  far  from  real  time. 
There  are  a  variety  of  ways  to  reduce  computational  requirements.  For  ex- 
ample, as  we  discussed  in  Section  2,  we  may  be  able  to  reduce  computations 
required  for  the  low-pass  filtering  operation  by  under-sampling  /lC^i,^). 
It  is  also  possible  to  increase  the  computational  speed.  For  example,  the 
computation  time  required  for  an  arithmetic  operation  is  becoming  smaller 
as  new  SP  chips  are  introduced.  In  addition,  the  algorithms  have  very  sim- 
ple structures  and  we  can  process  the  data  in  parallel  using  more  than  one 
SP  chip. 

In  summary,  our  preliminary  study  indicates  that  real  time  implemen- 
tation of  the  algorithms  discussed  in  previous  sections  at  a  reasonable  cost 
is  not  a  simple  task  due  to  a  very  large  amount  of  data  involved.  However, 
it  appears  to  be  within  a  reachable  goal  with  the  hardware  technology  that 
is  currently  available  or  will  shortly  be  available. 

TABLE  I 


Algorithms 

Choice  of  c  and  p 

c  =  100  nsec 
p  =  25% 

c  =  100ns 
p  =  50% 

c  =  200  nsec 
p  =  25% 

c  =  100  nsec 
p  =  50% 

Adaptive  Filtering 

iV  =  512,M  =  5,72  =  30 

Adaptive  Filtering 

iV  =  512,M  =  5,R=  1 

90  sec 
3  sec 

45  sec 

1.5  sec 

180  sec 
6  sec 

90  sec 
3  sec 

Modified  Adaptive  Filtering 
iV  =  512,Af  =  5,1  =  5,  R  =  30 

180  sec 

90  sec 

360  sec 

180  sec 

Modified  Adaptive  Filtering 
N >512,M  =  5,L  =  5,fl  =  l 

6  sec 

3  sec 

12  sec 

6  sec 

Temporal  Filtering 
N  =  512,72  =  30 

9  sec 

4.5  sec 

18  sec 

9  sec 
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